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1. INTRODUCTION 
The technique of conjugate gradient is useful in finding the problem’s minimum value. Consider this 
formula: 


Min f(x), xeR* (1) 


where f :R" —R is smooth [1]. The iteration technique is often used to solve Equation (1) and it is 
written as: 


Xp, =X, +a,d, , (2) 
where @, is the step length. In any complete conjugate gradient method, the step length @, is regularly 
chosen to satisfy the firm line search conditions. As the most eminent line search conditions, Wolfe line 


search necessitates that: 


f(x, +@,4,) S f(x) + 6a, g/d, (3) 
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dj g(x, +a,d,)20d; 8, (4) 


where 0 < 6 < oO. More details are found in [2]. Therefore, this study will focus on the CG method whose 
direction of search is written as: 


din =-8int By dy, (5) 


fi, is a scalar known as the CG update and g,,, is gradient. Different CG techniques chiefly convert into 


varied selections for the CG update, as reviewed in [3]. 

In convergence properties, the conjugate gradient method proposed by Dai and Yuan [4] is generally 
believed to be one of the best CG techniques. The CG update of the Dai and Yuan (DY) technique is given 
by: 


T 
DY — Sxu8eu 

k ~ T 6 

k Mk (6) 
where Y, = 814, — 8, - 


We utter that the search directions d,,, satisfy the sufficient descent condition if: 


Badd ssc lecull (7) 


where C is a positive constant. More details on performance profile are given in [5]. Now we make 


extension of the Dai - Yuan conjugate gradient technique and analyze its convergence. 


2. EXTENSION OF THE DAI - YUAN CONJUGATE GRADIENT TECHNIQUE 
Attempting to make an extension on the CG update suggested by Dai and Yuan through multiplying 


(5) with 2,,, and using (6), this equation is obtained: 


T 
T ScuSku 7 
Sinden = ee rd (8) 
di Vx 
which with (6) gives an equivalent formula to (6): 
T 
B = Bird gat 
aa (9) 


T 
8x 
For more details can be found in [6]. Obviously, if the Hessian matrix of f is non negative definite, the 


majority efficient search direction at x, will be the Newton direction: 


diy = —V" f (Xp) Ss (10) 


The Hessian matrices satisfy the secant equation: 
2 
VF Ce) Se = Ve (11) 


where S, = X,,, —X,. 
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Suppose that J denotes the identical matrix and assume that a matrix Vf (x, 41) is desired with 


a simplified construction satisfying the secant equation. Specifically, this formula is desired: 
After some algebraic manipulations one obtains : 


VEU Nests (12) 


So, the equation of secant is formed as follows: 


Mes Se = Ve (13) 


This equation describes the most popular spectral gradient method for optimization with the 
direction of search: 


1 
daw =— —— Ben (14) 


Uren 


where d,,,, which is defined in (14), would be replaced by d,,, in (8), the following equation is obtained: 





1 i 
p= pedo (15) 
Newt 8 Fe 


If a step - length @, is inexact, then: 


1 ee 1 
PB, Sy SE ORL ORE: (16) 
Nhat yi. d, 


Denoted this formula procedure as Be ae Definitely, more sufficient test would be required 77,,,. The 


coefficient 77,,, plays an important role in the current method. It can be computed by the formula: 


Va = 2| Ra Te + Sia; \ yy Se (17) 


This coefficient defines the most popular Yuan method [7]. The Extension algorithm is presented below and 
called Algorithm BDY. 


Algorithm BDY 
: Given constants € >0, 6 € (0,1), o €(6,1) and x, € R" . Let d) =—g,, k=0. 
: Examine a stopping criterion. If yes, then stop; if not, continue with the next step. 


: Define a step length @, using a proper line search. 


BPY by (16). 


0 

1 

2 

3:Let x,,, =X, +a@,d,, andcompute /, 
4 : Compute the search direction d,,, =—8,,, +f, d,.- 
5: Set k =k +1 and continue with step 1. 


3. CONVERGENCE ANALYSIS 
In this paper, f(x) is assumed as that : 


1. Fora given x, € R”, the function f(x) has lower bound on ‘Y = ix ER": f(x)< f(Xp)}- 


2. A positive constant L exists; so: 





A descent extension of the Dai - Yuan conjugate gradient technique (Basim A. Hassan) 


664 0 ISSN: 2502-4752 








|g)-gQ)|<Llx-yl], Vx, ye ® (18) 
where @ € 'Y is an open convex set. That is to say, the gradient g(x) is Lipschitz continuous in M . More 


details are found in [8, 9]. 


Theorem 1 
Suppose that the search direction d,,, is created by (5), and 77,,, <1. Then the descent condition 
holds, i.e., 











6 ides <0 (19) 
Proof: 
: F : . : T. 2 
This theorem is proved by induction. As d)=—g,, there is god) =-|g,| <0. 
Assume that gd , <0 for all k En. Through multiplying (5) by g,,,, the following equation is 
obtained: 
T d _ 2 lgeall T d 
Ste = — |g, +1 Fp Site 
yd, 
T 
2 Sid 
= ee - 1+ 7h z (20) 
yy 
T T 
7 2] Me Sand, — Vy 
= [8 el F 
yy 


Since yd, = 8.4, — 8, d,. then: 








Hee ad - gid +d 
Bada = [uf | aint lat 
T T 
= gual ies oe ad 21) 
keomk 
= lerel 





T lea -1)g;4.4, +"d,| 
y,4, 


From above equation we get: 








lgsall a. deat DY 
= as ele = B (22) 
y, 4, lone =d) ead; + g/d, | . 
Now, 
2 

BDY _ leu _ DY 

k i k+l T “ "Tk+l hk 
Ve d, (23) 
Wes e ates 





len = 1) B, ade + gd, 


Indonesian J Elec Eng & Comp Sci, Vol. 16, No. 2, November 2019 : 661-668 


Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 im) 665 





From above equation we get : 


T 
d 
Pe ge (24) 
gd, 


where @,,,=7,.,/1+(7,,, -Dl,) and 1, = 9),,d,/ 9, d, . Assume that gid, <0. From J, and 
Wolfe state g{,,d, > og/d, we get: 


sg: = fs ol (25) 
and 

Oe SM A+ Me —DO) (26) 
where 77,,; (1+ (77,,, —D)o > 0. The above equation to get : 


1 





Sida pe e455 0 (27) 


k+l 
Therefore, the proof is complete. 


Due to playing an important role in analyzing the convergence property for conjugate gradient, 
Zoutendijk’s condition [10] will be proved to be a part of the proposed Algorithm in this study. 


Lemma 1 
Let that d,,, is generated by (5) and step size @, fulfills (3) and (4), if f(x) satisfies the 


Assumptions, then : 


x (as ) < 0 


(28) 
ies léeall 


holds. 


Theorem 2 
Let that Assumptions | and 2 are held, and that the search direction d,,, is calculated by the 


formulation in the second step, then: 


a inf iran } =0 (29) 


Proof: 
Now, equation (29) can be proved by contradiction, assuming that there is a constant vy > 0 such 
that : 


lgnuz7. Vk 20 (30) 


Since d,,, =—8,,,+ 8, d,, then the result is the following equation: 


deat |’ = (f, )*|d, |’ = 20, Bis ft leenll (31) 
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Dividing both sides of (31) by (d Pee “oe and using (24), the following formula is obtained: 








Ia.al . la, | 2 Iga 
(di®ea) i (dj 84) CHa pe Cit 
lI (deel 1) 1 
Se | (32) 
Gay \dinsea Weal) leral 
Md 
“(at 8) “Teva” 


which, along with (26), can yield: 
| | <1 (33) 


Using (32) and (33), the following equation can be obtained: 





fil 

















(34) 
2 2 
(a8) (4, 8,) lea | 
: - s 2 T 2 
Using (34) recursively and noting that ld, | =-d, 2,= lg: ; 
2 
d kel] k 
Meal >< >< (35) 
(dia 8en) il gin | ‘4 
Then, from this equation and (30), the following formula can be derived: 
d 
( Cs ~ a (36) 
aul 
and this denotes that: 
pet 2 
> (Sider) aa (37) 


2 
it [eu 


This is in contrary to the Zoutendijk condition (28). Thus, (29) holds. 


4. NUMERICAL RESULTS AND DISCUSSION 

In this paper, the algorithm of extension was tested and compared to the method of Dai-Yuan [4]. 
For comparing methods employed in this study with other classical ones, the gradient errors will be checked 
for measuring the criteria of stop for algorithms. In general, the iteration will be forced to stop when the 
gradient norm satisfies this inequality: 


Si|< 10° (38) 
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By employing the process of Wolfe line search, the parameters will be selected as follows: 
0, =0.001 and 6, =0.9. (39) 


In this study, experiments were conducted on 30 problems of unconstrained optimization test of 
Andrei’s collection with the dimensions 100 and 1000; for details see Andrei [11]. Optimization problems is 
an important tool in all papers optimization [12-15]. The comparing data contain number of iterations (IN), 
the number of restart (NR) and the number of function evaluations (NF) (Table 1). Using Fortran 90 to code 
this methods. 


Table 1. Comparing different conjugate gradient methods with different test functions 














P.No 7 DY algorithm BDY algorithm 
ot NI NR NF NI NR NF 
1 100 27 8 14 26 7 13 
1000 26 8 13 25 7 13 
2 100 28 7 11 25 5 9 
1000 49 15 23 48 14 22 
3 100 125 29 83 125 29 83 
1000 629 106 393 550 90 342 
4 100 21 5 10 13 3 6 
1000 29 i 15 26 7 13 
5 100 61 15 40 61 11 37 
1000 101 27 64 79 22 49 
6 100 19 6 10 19 6 10 
1000 35 12 22 19 6 10 
7 100 151 26 79 137 24 73 
1000 156 28 85 153 25 81 
8 100 28 8 10 28 8 11 
1000 33 10 15 29 11 13 
9 100 59 12 31 47 10 25 
1000 51 10 26 54 11 28 
10 100 173 37 111 163 33 105 
1000 767 160 493 583 117 373 
11 100 133 26 85 126 25 83 
1000 381 71 243 347 69 229 
12 100 136 24 87 136 24 87 
1000 421 58 250 400 63 241 
13 100 47 11 30 53 4 29 
1000 62 15 37 62 14 36 
14 100 33 9 17 40 11 20 
1000 48 12 22 35 9 16 
15 100 60 11 35 56 12 36 
1000 55 14 35 54 8 31 
Total 3944 787 2389 3519 684 2077 





Problems numbers indicant for : “1. is the Extended Beale, 2. is the Penalty, 3. is the Perturbed 
Quadratic, 4. is the Extended Tridiagonal 1, 5. is the Generalized Tridiagonal 2, 6. is the Extended 
Himmelblau, 7. is the Extended Powell, 8. is Extended Cliff, 9. is the Extended Wood, 10. is the Quadratic 
QF2, 11. is the DIXMAANE (CUTE), 12. is the Partial Perturbed Quadratic, 13. is the Broyden Tridiagonal, 
14. is the LIARWHD (CUTE), 15. is the Generalized quartic GQ2.”. 

The summary of our results in Table 2. The result presented in Table 2 imply that BDY method 
improved over the performance of DY method. The improvement of method BDY over DY is 13 %, in 
average, of the iterations number, and 13 %, in average, of the restart number and 10 %, in average, of 
function evaluations number. 


Table 2. Ratio of algorithm BDY cost to DY cost 








NI NR NF 
DY algorithm 100 % 100 % 100 % 
BDY algorithm 86.94 % 86.91 % 89.22 % 
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5. CONCLUSION 

An extended method of nonlinear DY conjugate gradient was proposed in this paper. The extended 
direction refers to the descent direction. Analysis of convergence analysis was carried out and the statistics 
showed that the proposed technique is efficient for the given problems of nonlinear optimization test. 
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